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FINAL  REPORT  -  FA9550-10-1-0196 

NSSEFF-  INFORMATION  ACQUISITION,  ANALYSIS  AND 
INTEGRATION 


GUILLERMO  SAPIRO 

Department  of  Electrical  and  Computer  Engineering 
Duke  University,  Durham,  NC  27707 
guillermo.sapiro@duke.edu 


1  OVERVIEW  OF  ACCOMPLISHMENTS 

As  detailed  in  the  next  sections,  we  have  accomplished  all  the  challenges  in 
the  original  proposal  and  beyond.  We  have  advanced  science  in  all  directions 
we  targeted,  we  have  received  some  of  the  most  prestigious  awards  in  our  dis¬ 
cipline,  graduate  numerous  students  thereby  enhancing  the  US  work  force, 
and  contributed  to  science  dissemination  in  multiple  directions.  Particulars 
are  given  in  the  next  sections,  and  is  virtually  impossible  to  enumerate  all 
the  results  that  were  made  possible  by  this  NSSEFF  award.  At  the  science 
level  we  can  exemplify  just  a  few: 

1.  Developed  some  of  the  more  advanced  theoretical  and  computational 
results  in  the  important  topic  of  graph  matching. 

2.  Developed  new  cameras  and  computational  reconstruction  frameworks 
for  advanced  video  and  image  acquisition  with  unique  integrations  of 
sensing  and  processing. 

3.  Developed  advanced  computational  techniques  for  some  of  the  most 
critical  tools  in  signal  processing,  from  sparse  modeling  to  matrix  fac¬ 
torization. 

4.  Developed  theoretical  foundations  for  successful  algorithms  such  as 
deep  learning. 

5.  Solved  elegantly  old  problems  like  image  and  video  debluring,  intro¬ 
ducing  new  revolutionary  approaches. 
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6.  Introduced  new  fundamental  signal  representations,  ranging  from  new 
aspects  of  the  Weyl  transform  to  active  selection  of  signal  representa¬ 
tives. 

7.  Combined  fundamental  tools  such  as  random  forest,  hashing,  and  sub¬ 
space  learning  to  produce  state-of-the-art  results  in  numerous  classi¬ 
fication  tasks  at  a  fraction  of  the  computational  and  memory  cost  of 
competing  approaches. 

8.  Introduced  new  concepts  of  sparse  modeling  and  compressed  sensing 
into  shape  analysis  and  representation. 

9.  Demonstrated  how  fundamental  problems  like  gaze  analysis  can  be 
solved  with  cameras  that  cost  about  5  orders  of  magnitude  less  than 
current  technology. 

10.  Connected  sparse  modeling  and  compressed  sensing  with  the  funda¬ 
mental  theories  of  universal  modeling  and  minimal  description  length. 

11.  Provided  unique  theoretical  foundations  for  multimodal  signal  process¬ 
ing. 

12.  Developed  fundamental  theoretical  and  practical  results  on  Gaussian 
mixture  models  and  its  connections  with  compressed  sensing  and  sparse 
modeling. 

13.  Developed  novel  hierarchical  and  collaborative  tools  for  sparse  model¬ 
ing  and  compressed  sensing. 

14.  Made  contributions  in  applications  covering  health,  consumers,  and 
defense. 

These  are  just  examples  of  the  contributions,  which  resulted  in  also  nu¬ 
merous  technology  transfers  to  the  Department  of  Defense  (ONR,  ARO, 
NGA),  to  the  NIH,  and  to  industry  (Adobe,  LSS,  etc).  As  mentioned  above, 
these  also  resulted  in  numerous  awards  and  become  the  core  education  of  a 
large  number  of  PhD,  MSc,  and  post-docs. 

The  next  sections  provides  list  of  publications,  honors,  and  other  activ¬ 
ities  showing  the  accomplishments. 
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2  HONORS  AND  AWARDS 


2010:  Plenary  Speaker,  The  Learning  Workshop,  Snowbird,  April  2010. 

2010:  Plenary  Speaker,  SIAM  Image  Science  Conference,  Chicago,  April 
2010. 

2011:  Success  story  from  the  National  Geo  spatial- Intelligence  Agency  Basic 
Research  Program  (NURI). 

2011:  Helmholtz  Test-of-Time  Award,  International  Conference  Computer 
Vision,  “Geodesic  Active  Contours”  ICCV  ’95  paper. 

2012:  Best  Poster  Award,  P.  Sprechmann,  A.  Bronstein,  and  G.  Sapiro, 
“Real-time  online  singing  voice  separation  from  monaural  recordings 
using  robust  low-rank  modeling,”  International  Society  for  Music  In¬ 
formation  Retrieval  Conference,  Porto,  October  2012. 

2013  SIAM  Fellow. 

2013  Best  Paper  Award,  P.  Llull,  X.  Liao,  X.  Yuan,  J.  Yang,  D.  Kittle, 
L.  Carin,  G.  Sapiro,  and  D.  J.  Brady,  “Compressive  sensing  for  video 
using  a  passive  coding  element,”  Imaging  and  Applied  Optics  Congress, 
Arlington,  VA,  June  2013. 

2013  International  Society  in  Magnetic  Resonance  in  Medicine  Recogni¬ 
tion,  Top  5  cited  articles  in  Magnetic  Resonance  in  Medicine  10,  I. 
Aganj,  C.  Lenglet,  G.  Sapiro,  E.  Yacoub,  K.  Ugurbil,  and  N.  Harel, 
“Reconstruction  of  the  orientation  distribution  function  in  single  and 
multiple  shell  q-ball  imaging  within  constant  solid  angle”  Magnetic 
Resonance  in  Medicine  64:2,  pp.  554-566,  2010. 

2013  IEEE  Fellow. 

2013  Science  Advisory  Board,  Institute  for  Computational  and  Experimen¬ 
tal  Research  in  Mathematics  (ICERM),  Brown  University. 

2014  Plenary  Speaker,  European  Signal  Processing  Conference  (EUSIPCO), 
September  2014. 

2014-2017  Member,  National  Academies  Board  on  Mathematical  Sciences 
and  their  Applications  (BMSA). 
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2015  Plenary  Speaker,  CIARP,  December  2015. 

2015-2016  Distinguished  Israel  Pollack  Lecturer,  Techuion,  Haifa. 

3  PUBLICATIONS 

Journals  and  Book  Chapters 

1.  R.  Giryes,  A.  Bronstein,  and  G.  Sapiro,  “Deep  neural  networks  with 
random  Gaussian  weights:  A  universal  classification  strategy?,”  IEEE 
Trans.  Signal  Processing,  2016,  to  appear. 

2.  M.  Tepper  and  G.  Sapiro,  “Compressed  nonnegative  matrix  factoriza¬ 
tion  is  fast  and  accurate,”  IEEE  Trans.  Signal  Processing,  2016,  to 
appear. 

3.  E.  Elhamifar,  G.  Sapiro,  and  S.  Sastry,  “Dissimilarity-based  sparse 
subset  selection,”  IEEE  Trans.  Pattern  Analysis  Machine  Intelligence, 
2015,  to  appear. 

4.  A.  Thompson,  Q.  Qiu,  R.  Calderbank,  and  G.  Sapiro,  “Data  repre¬ 
sentation  using  the  Weyl  transform,”  IEEE  Trans.  Signal  Processing 
64:7,  pp.  1844-1853,  2016. 

5.  M.  Delbracio  and  G.  Sapiro,  “Hand-held  video  deblurring  via  efficient 
Fourier  aggregation,”  IEEE  Transactions  on  Computational  Imaging 
1:4,  pp.  270-283,  2015. 

6.  P.  Sprechmann,  A.  Bronstein,  and  G.  Sapiro,  “Supervised  non-negative 
matrix  factorization  for  audio  source  separation,”  in  Excursions  in 
Harmonic  Analysis  4,  2015. 

7.  G.  Sapiro,  “Mathematical  Image  Processing,”  in  The  Princeton  Com¬ 
panion  to  Applied  Mathematics,  N.  J.  Higham,  Ed.,  Princeton  Univer¬ 
sity  Press,  2015. 

8.  M.  Delbracio  and  G.  Sapiro,  “Removing  camera  shake  via  weighted 
Fourier  burst,”  IEEE  Trans.  Image  Processing  24:11,  pp.  3293-3307, 
2015. 

9.  V.  Lyzinski,  D.  Fishkind,  M.  Fiori,  J.  T.  Vogelstein,  C.  E.  Priebe,  and 
G.  Sapiro,  “Graph  matching:  Relax  at  your  own  risk,”  IEEE  Trans. 
Pattern  Analysis  Machine  Intelligence  38:1,  pp.  60-73,  2015. 
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10.  P.  Sprechmann,  A.  Bronstein,  and  G.  Sapiro,  “Learning  efficient  sparse 
and  low  rank  models,”  IEEE  Trans.  Pattern  Analysis  Machine  Intel¬ 
ligence  37:9,  pp.  1821-1833,  2015. 

11.  M.  Fiori  and  G.  Sapiro,  “On  spectral  properties  for  graph  match¬ 
ing  and  graph  isomorphism  problems,”  Information  and  Inference:  A 
Journal  of  the  IMA  4:1,  pp.  63-76,  2015 

12.  J.  Lucas  and  G.  Sapiro,  “Cancer:  What’s  luck  got  to  do  with  it?,” 
Significance  -  statistics  making  sense  12:2,  pp.  40-42,  April  2015. 

13.  I.  Aganj,  G.  Sapiro,  and  N.  Harel,  “Q-space  modeling  in  diffusion- 
weighted  MRI,”  Brain  Mapping-An  Encyclopedic  Reference ,  pp.  257263, 
Academic  Press,  2015. 

14.  Q.  Qiu  and  G.  Sapiro,  “Learning  transformations  for  clustering  and 
classification,”  Journal  of  Machine  Learning  Research  16,  pp.  187-225, 
2015. 

15.  J.  Yang,  X.  Liao,  X.  Yuan,  P.  Hull,  D.J.  Brady,  G.  Sapiro,  and  L. 
Carin,  “Compressive  sensing  by  learning  a  Gaussian  mixture  model 
from  measurements,”  IEEE  Trans.  Image  Processing  24:1,  pp.  106- 
119,  January  2015. 

16.  M.  Tepper  and  G.  Sapiro,  “A  bi-clustering  framework  for  consensus 
problems,”  SIAM  Journal  on  Imaging  Sciences  7:4,  pp.  2488-2525, 
2014. 

17.  J.  Yang,  X.  Yuan,  X.  Liao,  P.  Llull,  D.J.  Brady,  G.  Sapiro,  and  L. 
Carin,  “Video  compressive  sensing  using  Gaussian  Mixture  Models,” 
IEEE  Trans.  Image  Processing  23:11,  pp.  4863-4878,  2014. 

18.  M.  Fiori,  P.  Muse,  and  G.  Sapiro,  “A  complete  system  for  candidate 
polyps  detection  in  virtual  colonoscopy,”  International  Journal  of  Pat¬ 
tern  Recognition  and  Artificial  Intelligence  28:7,  November  2014. 

19.  J.  Kim,  C.  Lenglet,  Y.  Duchin,  G.  Sapiro,  and  N.  Harel,  “Semi¬ 
automatic  segmentation  of  brain  subcortical  structures  from  high-field 
MRI,”  IEEE  Transactions  on  Information  Technology  in  Biomedicine 
18:5,  pp.  1678-1695,  2014. 
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20.  G.  Prasad,  J.  Joshi,  N.  Jahanshad,  J.  Villalon,  I.  Aganj,  C.  Lenglet,  G. 
Sapiro,  K.  McMahon,  G.  de  Zubicaray,  N.  Martin,  M.  Wright,  A.  Toga, 
and  P.  Thompson,  “Automatic  clustering  and  population  analysis  of 
white  matter  tracts  using  maximum  density  paths,”  Neuroimage  97, 
pp.  284-295,  August  2014. 

21.  J.  Hashemi,  T.  Vallin  Spina,  M.  Tepper,  A.  Esler,  V.  Morellas,  N.  P. 
Papanikolopoulos,  H.  Egger,  G.  Dawson,  and  G.  Sapiro,  “Computer  vi¬ 
sion  tools  for  low-cost  and  non-invasive  measurement  of  autism-related 
behaviors  in  infants,”  Autism  Research  and  Treatment ,  2014. 

22.  B.  D.  Harrison,  J.  Hashemi,  M.  Wellington,  G.  Sapiro,  and  J.  Berman, 
“A  tetraploid  intermediate  precedes  aneuploid  formation  in  yeasts  ex¬ 
posed  to  fluconazole,”  PLoS  Biology,  March  2014. 

23.  M.  Tong,  Y.  Kim,  L.  Zhan,  G.  Sapiro,  C.  Lenglet,  B.  A.  Mueller,  P. 
M.  Thompson,  and  L.  A.  Vese,  “A  vectorial  total  variation  model  for 
denoising  high  angular  resolution  diffusion  images  corrupted  by  Rician 
noise,”  Methods  and  Applications  of  Analysis  21,  pp.  139-164,  2014. 

24.  L.  Zhan,  N.  Jahanshad,  Y.  Jin,  C.  Lenglet,  B.  A.  Mueller,  G.  Sapiro, 
K.  Ugurbil,  N.  Harel,  A.  W.  Toga,  K.  O.  Lim,  and  P.  M.  Thompson, 
“Magnetic  resonance  field  strength  effects  on  diffusion  measures  and 
brain  connectivity  networks,”  Brain  Connectivity  3,  pp.  72-86,  2013. 

25.  J.  M.  Duarte-Carvajalino,  C.  Lenglet,  K.  Ugurbil,  S.  Moeller,  L.  Carin, 
and  G.  Sapiro,  “Estimation  of  the  CSA-ODF  using  Bayesian  com¬ 
pressed  sensing  of  multishell  HARDI,”  Magnetic  Resonance  in  Medicine, 
Dec.  2013. 

26.  S.  Jbabdi,  J.  Xu,  J.  L.  Andersson,  S.  Moeller,  E.  J.  Auerbach,  M.  F. 
Glasser,  M.  Hernandez,  G.  Sapiro,  M.  Jenkinson,  D.  A.  Feinberg,  E. 
Yacoub,  C.  Lenglet,  D.  C.  Van  Essen,  K.  Ugurbil,  and  T.  E.  Behrens, 
“Advances  in  diffusion  MRI  acquisition  and  processing  in  the  Human 
Connectome  Project,”  Neuroimage,  October  2013. 

27.  E.  Caruyer,  C.  Lenglet,  G.  Sapiro,  and  R.  Deriche,  “Design  of  multi¬ 
shell  sampling  schemes  with  uniform  coverage  in  diffusion  MRI,”  Mag¬ 
netic  Resonance  in  Medicine,  June  2013. 
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“Deep  learning  with  hierarchical  convolution  factor  analysis,”  IEEE 
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